Skip to content

fix(lcm): recover searchable history with source-grounded summaries - #144

Merged
ScriptedAlchemy merged 24 commits into
mainfrom
fix/lcm-retrieval-evidence
Nov 17, 2026
Merged

ScriptedAlchemy merged 24 commits into
mainfrom
fix/lcm-retrieval-evidence

Conversation

@ScriptedAlchemy

@ScriptedAlchemy ScriptedAlchemy commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Problem

Agents could not reliably recover prior project evidence: summary search was limited to one thread, tool output was absent from text search, and some message hits had no readable timeline handle. Binary payloads consumed context, unrelated provider instructions could contaminate summaries, and failed summary passes could remain stalled.

Change

Add project-scoped summary search and tool-output search with bounded excerpts and exact source handles. Share text-preserving binary filtering across history retrieval and handoffs, isolate summary generation at provider boundaries, and recover stalled passes with bounded retries using the configured model.

The integration fixes also cover copied indexes, stale tool rows, cursor scope, peer attribution and long evidence ranges. CI exposed two further failures: oversized tokens overflowing a base64 regex, and FTS writes leaving 64-bit row IDs that Node tried to narrow despite discarding the result. These are fixed at the payload and SQLite boundaries.

Scope

Maintainer-directed LCM repair in this fork, opened at the user's explicit request. No new client UI or automatic model switching. Providers without established instruction isolation no longer offer summaries.

Verification

Integration audit at f31387e2034a

The follow-up preserves the selected Codex summary model even when model inventory does not list it, and preserves readable UTF-8 base64 when a wrapped line splits a multibyte character. Both defects were reproduced before the fixes.

This head integrates the reviewed heads of #142, #143 and #145 so their shared provider, group and memory behavior is checked together. Fact verdicts remain migration 115; summary and tool search move to 116 and 117. Known earlier search-preview databases are reconciled atomically, preserving indexes, data and migration timestamps. Preview collision and rollback regressions are covered.

  • 2,728 unique focused tests passed, plus 11 React compiler cases. This includes the 46 runtime cases behind the earlier CI failures.
  • All 42 affected-package compiler commands passed directly. Native build, dependency-patch validation, formatting, lint, both Knip scans and GraphQL checks passed.
  • The changed-file gate used --skip-tests after the selected files were run through their owning configurations: 67 of 68 selected files passed locally. The remaining web component test could not launch because Chromium was absent, and the standard installer returned invalid/truncated archives. The cached typecheck launcher also could not initialize its seccomp supervisor; the direct compiler results above cover that environment limitation.
  • Integrated UI verification remains unavailable because the required built-in T3 Browser tools are absent. Private replay and live-model claims below were not independently repeated in this audit. Fresh CI on this exact head is required before merge.

Earlier feature-branch verification

  • Final vp run check:changed passed on the unchanged repaired tree: 1,999 tests across 39 files, 37 affected-package typechecks, format, lint, knip, native-addon and GraphQL checks.
  • New payload and SQLite regression tests failed before the fixes. Afterward, payload/migration tests passed 24/24 and SQLite/runtime/group integration tests passed 275/275. Group tests now wait for persisted publication before asserting.
  • Full payload comparison after the regex repair: all 183,060 saved items produce identical filtered output, with zero errors.
  • Private-copy tool replay: 48 recorded searches, 88,070 indexed items and 374 valid handles, with zero scope, snippet or backfill errors. Separate actual MCP replay opened all 181 message hits.
  • Blind review of 78 outputs from 13 synthetic fixtures found one unsupported completion claim and one uncertain attribution among 240 statements. All 24 rollups fit 1,200 characters. This does not estimate production factuality.
  • Schema-pinned routing evals retained their original grades: Kimi 10/12, DeepSeek 6/12. Kimi's misses were grading limitations; DeepSeek's failures requested broad activity. Results used mocked tools and do not establish recall.

The agent-context-increase label explicitly accepts the measured cost of the new retrieval tools and evaluated guidance: +544 bytes in provider instructions and +1,626 bytes in the MCP tool list. Budget measurement remains enforced.

Limits

Relevance improvements are mixed, long all-term queries can miss evidence, and common queries require growing SQL work. Binary filtering remains heuristic; encoded text stays encoded. In the corpus, source changes invalidate 688 of 5,886 summaries; regeneration still requires successful future passes. No production regeneration or live settings changes were performed. Callers must expand original evidence to verify summary claims.

Implemented with GPT-6-Astra through the Codex harness, with Claude Opus 5.5 collaboration.

ScriptedAlchemy and others added 19 commits November 16, 2026 19:06
Fails on the current code: candidates are saved without a check, nothing
records a verdict, recall offers every saved fact, and there is no
checkFactStanding operation for the eval to run. `pnpm memory-standing-check
--no-check` shows the old behaviour failing the four set-aside cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… standing

After extraction, one more call reads only the proposed statements, from an
empty directory, and classifies each as standing, task or code. Every
candidate is saved with that verdict; recall offers only standing and
unchecked facts, through one SQL fragment, while search and the new Set aside
filter still show the rest. A failed check saves nothing and retries the batch
in 30 minutes, recording a failure after three attempts. Editing a fact's text
clears a check verdict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…estating it

Recall now skips a fact the message already states (#140), so the query shares
the fact's terms without repeating them in order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…statement

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent's edit or replacement keeps a fact's verdict; a person's edit clears it.
Recall's term counts and corpus size now cover only recallable facts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion

A keyring login keeps its home, because Codex keys it by that path, but loses
tools and developer instructions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ind evaluation

Text generation offers the standing check only where a runner can run blind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion isolation

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the text-generation model's provider cannot run the standing check
(Grok, Muse Code, Antigravity, some OpenCode and Pi setups), every batch
paid for extraction, failed the check and retried three times, saving
nothing. Ask the text-generation facade whether the selected instance
supports the check before extracting, and finish the batch as skipped
with a reason that names the setting to change. Transient check failures
keep the 30-minute retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng it

A message to everyone started every member at once, each told to give its own
view, so a group answered one question several times in parallel and then
traded agreements in reaction rounds.

A message to everyone now starts only the thread's own model. Its reply wakes
the others, who see the question and that answer and add only a correction, a
risk, evidence, or work nobody has done, or reply PASS. The lead owns the
answer: it can hand part of the work to someone with a final handle line, and
when someone adds something it says briefly what changes and closes with @user.
Naming members with @ still starts them all at once. A seat woken by the lead's
reply also receives the user's attachments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f31387e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Audit in progress on the newly pushed 612d4a90bf72c3f83a464b5e0cc27d6c9c974fe4. I independently reproduced the oversized-output and discarded-SQLite-metadata failures and am incorporating this commit's fixes rather than replacing them with parallel implementations.

Two further confirmed issues are being fixed in this pass:

  • Codex summary generation can substitute the account default when the selected model is absent from its inventory, while the summary retains the selected model in metadata. The existing summary regression failed on that substitution; the minimal exact-model fix passes the full Codex suite.
  • Readable UTF-8 encoded as base64 can be removed when wrapped lines of 4,096 characters split a multibyte character. The whole block is correctly recognized as readable, but the subsequent per-token scan reclassifies its partial lines as binary. The same wrapped-block/per-token structure is still present in this head.

I am also resolving integration with #142, including the shared provider isolation paths and migration 115 collision. The intentional context growth has been reviewed and accepted through agent-context-increase; no budget measurements were reduced.

Validation limitation: the repository's test-t3-app skill requires built-in T3 Browser preview tools and directs UI verification to stop when they are absent. Those tools are absent here. Private replay, live-provider probes and live-model factuality claims in the description have not been independently repeated. The final review will identify the exact validated head and remaining CI status.

Honor the selected Codex summary model and preserve readable wrapped UTF-8 payloads. Integrate the reviewed standing-memory, lead-first group, and conservative recall changes.

Keep fact verdicts at migration 115 and move search migrations to 116–117. Reconcile the known earlier search preview histories transactionally without rebuilding or discarding their search data.

@ScriptedAlchemy ScriptedAlchemy left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Completed the integration audit on f31387e2034ac8fe5dafc97d6b5340f3506acc69 and pushed the verified tree.

Confirmed fixes:

  • Codex summaries now use the exact configured model even when inventory/default selection would otherwise substitute another model.
  • Readable wrapped UTF-8 base64 is classified once as a complete block, so partial lines no longer remove readable evidence.
  • The #142 migration collision is resolved with fact verdicts at 115, summary search at 116 and tool search at 117. Known earlier preview histories upgrade transactionally without rebuilding or discarding their existing search data; unknown histories fail before writes and injected failure rolls back schema/ledger changes.

I retained the author's 612d4a90 payload-overflow, SQLite-metadata and publication-wait fixes, and integrated reviewed heads #142 c8d336fa, #143 1237645c, and #145 c6c6f661. Provider isolation, standing checks, lead-first groups, bounded context delivery and conservative recall all pass their combined coverage.

Validation: 2,728 unique focused tests plus 11 React compiler cases passed; all 42 affected-package direct compiler commands passed. The full changed-file scope also passed native build, unused-patch validation, format, lint, both Knip scans and GraphQL checks. Source hashes stayed unchanged through validation. The gate's cached compiler launcher failed before compilation because its seccomp supervisor is unavailable here; I ran the exact compiler commands directly. Its --skip-tests setting avoided repeating the owning-package runs.

Validation gaps, as requested: 67 of 68 gate-selected files ran successfully here. ProjectMemoryPanel.browser.test.tsx could not launch because Chromium is missing; one standard dependency-install attempt failed with invalid/truncated downloads. The repository's test-t3-app skill also requires built-in T3 Browser tools, which are absent, so the integrated UI demonstration could not be repeated. The private replay and live-model factuality/routing claims were not independently rerun. These limits are documented in the PR body. No production data, live provider settings, CI rules or budget enforcement were changed by the audit.

Ready for review; waiting for the complete fresh CI result on this exact head before merge.

@ScriptedAlchemy
ScriptedAlchemy marked this pull request as ready for review November 17, 2026 17:28
@ScriptedAlchemy
ScriptedAlchemy merged commit 9c8d669 into main Nov 17, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant